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ABSTRACT 
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Chapter Fifteen 



Using Web-Based Surveys to Conduct 
Counseling Research 

Darcy Haag Granello and Joe E. Wheaton 



Internet use in the United States is growing by more than two million 
new users per month and more than 150 million Americans have regular 
access to the Internet (Cyber-Atlas, 2002). Americans are becoming 
increasingly computer literate and the Internet is becoming the 
communication and information method of choice for many people (Duffy, 
2000). Because of the expansive growth of the online population, researchers 
from many disciplines are starting to see the benefits of collecting data 
over the Internet (Schleyer & Forrest, 2000). Consequently there has been 
a rapid proliferation of online data collection as well as research published 
from online data sources (Crawford, Couper, & Lamias, 2001). 

In spite of this increased use of the Internet for data collection, there 
is little published research about the process of data collection online. That 
is, discipline specific studies publish the results of their web-based surveys 
in discipline-specific journals but little information is available on the 
process of Internet-based data collection. Thus it is difficult for the researcher 
wishing to use this cyber data collection method to find resources to use as 
guides. 



Collecting Data over the Web 

The two most common types of Internet-based data collection are e- 
mail surveys and web-based surveys. With e-mail surveys, the participant 
receives an e-mail with a survey embedded in it. To complete the survey, 
the participant “replies” to the message and fills in the information just as 
is done with a paper and pencil survey. The survey is then e-mailed to the 
researcher who transfers the raw data into a database. The advantage of the 
e-mail survey is minimal technology or computer literacy is required by 
the user. However, a major disadvantage is the loss of anonymity as the 
user’s e-mail address is transmitted with the answers. 

The use of a web-based survey, on the other hand, requires the ability 
to use a web browser but allows for flexibility and design control, automatic 
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data entry, and anonymity (Harris & Dersch, 1997). With web-based surveys 
participants are solicited to participate in the survey either by traditional 
mail, e-mail, telephone, or through other websites. Participants are given 
access information to enter the survey website; they complete the form 
online and then click on a “submit” button when they have completed their 
response. 

Web-based surveys are quickly becoming the online data collection 
method of choice by researchers. This chapter outlines the advantages and 
limitations of web-based data collection and then describes the design and 
implementation of a web-based survey that has been conducted by the 
authors. Practical suggestions and lessons learned are included for 
researchers wishing to engage in their own online data collection. 

Advantages of Web-based Surveys 

Web-based surveys have several important advantages over paper and 
pencil surveys that make them particularly attractive to researchers. These 
advantages include reduced time, lowered cost, ease of data entry, flexibility 
and control over format, software development, recipient acceptance of the 
format, and the ability to obtain additional response-set information. 

Reduced time. A primary advantage of web-based surveys is that they 
dramatically decrease response times (Lazar & Preece, 1999). Farmer (1998) 
reported that typical turnaround time is four to six weeks with traditional 
mail surveys, two to three weeks for telephone surveys, and only two to 
three days for web-based surveys. Franceschini (2000) also noted reduced 
turnaround time. In his study, half of the respondents were sent mail surveys 
and the other half were surveyed via the Internet. He reported that 21 of the 
29 web-based responses were received before there were any responses to 
the traditional mail survey. In fact, Bauman, Airey, and Atak (1998) found 
that the majority of recipients of an e-mail survey either responded within 
one to two days of receiving the initial solicitation or not at all. 

Lowered cost. Web-based surveys can have substantially lower costs 
than traditional mail surveys because there are no printing, postage, or 
stationary costs (Bauman, Airey, & Atak 1998). There are, however, some 
costs associated with web-based surveys, primarily for prograrmning, using 
space on a server and some limited data entry. Farmer (1998) argued web- 
based surveys are 50% less expensive to implement than telephone surveys 
and 20% less expensive than mail surveys. In their study, Schleyer and 
Forrest (2000) found the web-based survey to be 38% less expensive than 
mail surveys. If there are technical problems with the survey, however, 
costs associated with “help desk” technicians can increase costs dramatically 
(Couper, Blair, & Triplett, 1998). To address the need for help desk support. 
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an initial piloting of the survey is essential to reduce the number of 
unforeseen technical problems that can increase cost. 

Rase, of data entry. In traditional paper and pencil surveys or even 
with e-mail surveys, data entry can be extremely expensive, time consuming, 
and subject to error. An electronic survey can be configured to send data to 
a database, spreadsheet, or a text file for use with a statistical package, 
eliminating the need for data entry with all its attendant problems. 

Flexibility and control over format . Using the web allows researchers 
to use flexible design formats such as color, graphics, innovative question 
displays, split screens, embedded programs (applets), animation and sound 
(Dillman, Tortora & Bowker, 1999). Additionally, researchers can control 
the order in which respondents answer the questions easier than with paper 
and pencil surveys where respondents can flip back and forth and change 
answers (Wyatt, 2000). Other rules, such as “select one answer only” or 
“do not leave this question blank,” can be enforced with radio buttons (Lazar 
& Preece, 1999). With web-based surveys, the order and formatting of 
questions can be easily altered, which is particularly useful for Delphi studies 
(Wyatt, 2000). One study found that dropout rates were significantly lower 
when the order of data collection was changed and demographic information 
was collected at the beginning of the survey (drop-out rate 10.3%) rather 
than at the end (drop-out rate 17.5%) (Bosnjak & Tuten, 2001). Although 
this flexibility can be extremely useful, there are no definitive answers as 
to the psychometric effects of the various web-based formatting options 
(Amau, Thompson & Cook, 2001). 

Software development . Most Internet surveys are now constructed 
using HTML format with the potential respondent contacted via an e-mail 
cover letter. HTML editors are becoming increasingly more sophisticated 
and easy to use and data can be captured by a program on the server called 
a Common Gateway Interface (CGI) script. Several products exist that 
provide both the editing capacity for HTML and the necessary CGI scripts 
for capturing data. The most common of these are Microsoft’s FrontPage 
and Macromedia’s ColdFusion (Solomon, 2001). Additionally, there are 
some software programs designed specifically for web-based surveying 
that offer additional features such as management of the distribution of e- 
mail cover letters, built-in statistical analysis, the ability to generate reports, 
and automatic tracking of respondents. Examples of these programs include 
Zoomerang, Perseus’s Survey Solutions for the Web, Creative Research 
System’s The Survey System, and Survey Said™ Survey Software 
(Solomon). 

Recipient acceptance of the format. There is some evidence that the 
Internet is becoming more acceptable to respondents as a method of 
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collecting data, particularly for males (Dillman et al., 2001) and for 
individuals who are college-educated (Cartwright, Thompson, Poole, & 
Kester, 1999; Franceschini, 2000). Several authors have noted that self- 
disclosure is increased when people communicate via the Internet (e.g., 
Joinson, 1999; Joinson, 2001; Moon, 2000). Conboy, Domar, and O’Connell 
(2001) found that an Internet survey offered the necessary assurances of 
anonymity to allow respondents to give accurate data surrounding very 
sensitive health issues. Participants appear to accept claims of confidentiality 
and anonymity, even though there is a possibility that Internet password 
and encryption codes could be broken (Harris & Dersch, 1 997) and Internet 
Protocol (IP) addresses can be identified. In fact, IP addresses that can 
identify a specific computer are easily obtained (it is merely a check box in 
Microsoft’s FrontPage). However, identifying who actually used the 
computer could be more difficult, depending on the situation. 

Ability to obtain additional response-set information. With traditional 
paper and pencil surveys researchers only can know the results of the 
responses. Using the Web, researchers can learn about the respondents’ 
answering process (Bosnjak & Tutan, 2001). For example, researchers can 
identify the number of people who viewed the survey compared with those 
who completed it, or if the software will allow, the number of people who 
started the survey but did not complete it (Bosnjak & Tuten). Information 
such as time of day or day of the week of the response also can be tracked. 

Limitations of Web-based Surveys 

In spite of the many advantages of web-based surveys, concerns about 
their use have been raised in the literature. These concerns focus on the 
following limitations: representativeness of the sample, response rates, 
measurement errors, technical difficulties, and accessibility issues for 
persons with disabilities. 

Representativeness of the sample. Internet use in the United States 
continues to grow. In one month alone (September, 2001), 143 million 
Americans (54%) used the Internet, representing a 26% increase over August 
of 2000 (CyberAtlas, 2002). In spite of this growth, access to the Internet 
remains unequally distributed over the U.S. population. Most Web users 
are white (87.2%), male (66.4%), married (47.6%), and highly educated 
with almost 88% having some college and over 59% having obtained at 
least one degree. Additionally, 48% of Internet users are 35 years old or 
younger (Graphics, Visualization, and Usability Center, 1999). There are, 
however, recent reports that suggest the demographics of Internet users are 
becoming more inclusive. Annual growth rates for 1999 and 2000 were 
25% for Internet use by individuals in the lowest income households (less 
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than $15,000 per household per year). Additionally, Internet use among 
African Americans increased at an annual rate of 33% during 2000 and by 
30% for Hispanics (CyberAtlas, 2002). Nevertheless, it remains the 
responsibility of the researcher to insure that all members of a defined 
population have equal access to the technology needed to complete the 
survey (Dillman, Tortora, Conradt, & Bowker, 1998). To the extent that 
certain portions of a population are excluded, the generalizability of the 
survey is compromised. 

Response rates . Several studies using web-based surveys have found 
lower response rates than traditional mail surveys (Medin, Roy, & Ann, 
1999; Nichols & Sedivi, 1998). However, unless the web-based survey 
uses a sampling method that allows only certain individuals to access the 
survey, it is impossible to know the response rates. For example, when 
participants for electronic surveys are recruited via newsgroups or search 
engines, researchers are not able to pinpoint the number of individuals who 
received the information and therefore cannot determine response rates or 
speak to the representativeness of the sample (Schleyer & Forrest, 2000). 
To circumvent this difficulty many web-based surveys make use of an initial 
e-mail to a targeted group that contains a specific URL to access the survey. 
This e-mail also can include an access code, password, or PIN to ensure 
that only those who have been targeted can complete the survey and to 
prevent any individual from completing the survey more than once (Wyatt, 
2000). Researchers are cautioned, however, not to make the web-based 
survey too difficult to access with too many codes and passwords, as this 
added complexity can lower response rates (Cartwright et al., 1999). 

To reduce the problem of lowered response rates, several researchers 
have advocated a system of multiple reminders. This can be done easily if 
the original solicitation was done via e-mail. Kittleson (1997) found that it 
was possible to double response rates with e-mail follow-up reminders, but 
others have claimed more modest success (Solomon, 2001) 

Measurement errors. Very little is known about the psychometric 
implications of moving a survey from traditional paper and pencil to an 
electronic format (Amau et al., 2001). Bowker and Dillman (2000) found 
that the placement of the electronic survey on the page (left-aligned, right- 
aligned) affected the respondents’ reactions to the survey, with some who 
received the right-aligned survey stating they were confused and rating the 
design unfavorably, although the placement did not affect response rates. 
Wyatt (2000) cautioned that the effects of design choices must be 
investigated, noting that in translating the survey from paper to electronic 
formats, items can be perceived differently by participants, thus affecting 
the validity of the survey. For example, not scrolling down to see an entire 
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list of options in a list box or not understanding how to correct a mistaken 
response could affect survey results. Others argue that as long as the 
electronic survey format is similar to paper and pencil surveys, traditional 
surveys can appropriately be transferred to the web (Lazar & Preece, 1999). 

Technical difficulties . Not everyone who completes a web-based 
survey will be extremely computer-literate, nor will everyone have access 
to the most up-to-date technology. Dial-up access is still the most popular 
method to access the Internet (80%; CyberAtlas, 2002), and 66.5% of 
Internet users have a connection speed of 56k or slower (Graphics, 
Vizualization, and Usability Center, 1999). As of April 2002, the most 
commonly used Internet browsers were Internet Explorer (90%, versions 
4.x and above) and Netscape Navigator and related compatible products 
(5%, versions 4.x and above) (W3Schools.com, n.d.). Researchers using 
web-based surveys must ensure that their pages are easily downloaded and 
maintain their formatting in all types of software and hardware 
environments. Additionally, formatting issues such as open-ended questions 
or questions arranged in tables can lead to higher drop-out rates, as can the 
absence of clear navigational aids (Bosnjak & Tuten, 2001). Notably, one 
study found that although sophisticated formatting can make the survey 
more attractive and interesting, surveys with advanced features and 
sophisticated designs (HTML tables, multiple colors, motion, sound) had a 
5% lower response rate than simple surveys (black letters on a white screen) 
(Dillman et al., 1998). The authors noted that the sophisticated design was 
slower to load, particularly on older browsers, and some older browsers 
were more likely to crash when attempting to load the survey. It took 
respondents of the sophisticated questionnaire more than twice as long to 
complete as those who used the plain questionnaire. In another study, 
researchers developed a complex web-based survey. They found that 
although 523 potential respondents said they had access to the Internet, 
only 73 of those had the capability to respond to this technologically 
sophisticated survey (Nichols & Sedivi, 1998). The capacity of browsers 
to handle complex designs has undoubtedly improved since these studies. 
Nevertheless, although much more research clearly must be done, it still 
appears that there is a point at which adding more features to the surveys 
becomes self-defeating to response rates. Piloting the survey with a 
representative sample of the population and on a wide variety of computing 
formats should help to reduce these difficulties (Wyatt, 2000). 

Lack of accessibility for persons with disabilities. Printed material 
presents problems for persons with visual impairments (Job Accommodation 
Network, 2002) and web pages can present special problems unless they 
are properly designed. Surveys are rendered as forms in HTML and forms 
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can be handled well by recent versions of computer screen readers (e.g., 
JAWS from Freedom Scientific) or special software designed to read web 
pages (e.g., HomePage Reader from IBM). Many web pages were not 
created with accessibility in mind (Wheaton, Chovan, O Briant, & Howell, 
2001). However, resources exist on designing accessible web pages (e.g., 
the Web Accessibility Initiative (2002); World Wide Web Consortium, 1999). 
As Wheaton et al. noted, most problems are easily repaired. 

Web page authors should provide information about the survey through 
a link from the home page. Such information should be linked from the top 
of the home page and be written in straight text. It should include information 
about the structure of the site by telling readers what is included in the 
survey (e.g., ^^The survey contains multiple choice and fill in the blank 
questions”), navigation tips (“Move between the fields by hitting the Tab 
key”), and how to contact the authors if help is needed. A simple and effective 
service is to provide a help line where the person can call and the form can 
be filled out. No names need to be given for this service so anonymity is 
maintained. It should be noted that web-based forms are more accessible 
than paper forms (which are just pieces of paper to persons with no vision) 
because the assistive software can read them as long as simple web design 
templates are followed. 



Sample Study 



Researchers wishing to use web-based surveys must consider the 
advantages and limitations addressed above in order to determine whether 
their research needs can be met with a Web-based survey. In this section of 
the chapter, we will discuss a sample study using a Web-based survey with 
recommendations for designing, implementing, and analyzing the results. 
The study was conducted by the authors in the autumn of 2000 (Wheaton 
and Granello, 2001) and distributed to employees of a large state agency, 
which had its own e-mail system. Participation was voluntary but strongly 
encouraged by the agency administration as the results would be used to 
determine the training needs for all staff over the next three years. 

Study Design 

In this study, we wanted to ascertain the training needs of all 
employees at the state agency. To accomplish this, we wanted to develop a 
Web-based instrument that would be easily understood and accepted by the 
recipients, allow for questions in a wide variety of formats (e.g.. Likert- 
type scale questions, rank ordering questions, and open-ended questions), 
and be tailored to meet the needs of the various employees of the agency. 
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Although all participants were employees of the agency, there were really 
four different groups of employees, based on major job classifications. The 
four groups had both overlapping and discrete training needs. Thus, there 
were really five sets of questions that needed to be asked, one set that was 
common to all participants and four additional sets that were specific to 
each group. In addition, we wanted to conduct follow-up phone interviews 
of volunteers, but we wanted to ensure the anonymity of those volunteers. 
Therefore, we could not ask for identifying information on the survey. 

Development of the Instrument 

We chose to develop the questionnaire in Microsoft FrontPage, a 
commonly used and widely available web page authoring tool available in 
the Microsoft Office suite of programs. We also chose FrontPage because 
we had used it on many other occasions and were familiar with the product 
and the construction of forms within it. FrontPage allows for up to 256 
questions, so it was ideal for our purposes. 

The final instrument consisted of 166 questions. Sixty-four questions 
were in the general section and 13, 43, 16, and 30 questions in the specific 
sections for the four groups, therefore no person had to answer all 166 
questions. The number of possible questions for each person ranged, 
depending on employee classification status, from 77 to 107. Participants 
were instructed to answer the first 64 questions and then click on one of 
four internal links, one for each of the four employee groups. Clicking on a 
link took the user to the appropriate section. When they reached the end of 
the questions for their employee groups, participants were asked to click 
on a submission button, which caused the form to be submitted. When the 
form was submitted, the server recorded the data, the time of day, the date, 
and the Internet Protocol (IP) address of the computer submitting the form. 
When the computer had recorded the information, a confirmation messages 
was sent back to the sender. 

We included the information on time, date, and IP address in order to 
check for multiple entries, not as a means of identifying the respondents. 
When piloting the survey, we found that there was no method available to 
prevent a person from accidentally clicking the submit button more than 
once, thus submitting the data more than once. This inadvertent error could 
easily occur during peak times on the Internet because sending the data 
took some time, even on a high-speed connection, and the confirmatory 
response added to the delay. When such delays occurred, respondents might 
easily have believed that they had not submitted the form properly and 
click again in order to resubmit. Adding the date, time, and IP address served 
as a cross check of the submission. If a person submitted the form twice. 
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the date and IP address would be exactly the same, and the times of 
submission typically would be within a minute of each other. As a final 
check, all the answers would be exactly the same, as it was the same form 
that was being submitted every time. Identifying such errors was easy. We 
simply opened the data in a spreadsheet program and sorted by IP address 
and time of submission. If duplicates existed, the IP addresses matched and 
the times would be within one or two minutes of each other. By collecting 
information on date and time of submission, we also were able to examine 
response patterns by time of day and day of the week. 

There are two important data formatting issues that should be noted: 
data were saved as tab-delimited text and the names of the variables followed 
SPSS conventions (they began with a letter and were no longer than eight 
characters). Although FrontPage allows for the creation of a database that 
data can be saved to directly, we chose to examine the data with Microsoft 
Excel and SPSS. The tab-delimited format allowed the data to be quickly 
imported into both programs. To move the data into Excel, we opened the 
text file, chose “Edit” from the Menu Bar at the top of the page, chose 
“Select All” from the Edit menu, and then selected “Copy.” We then opened 
Excel, placed the cursor in the A1 cell and chose “Paste” from the Edit 
Menu (or clicked on the Paste icon). To open the data in SPSS, we chose 
“Read Text Data” from the file menu (SPSS version 11.0) and checked the 
appropriate boxes that identify tab-delimited text. In both cases the variable 
names were saved and the data was automatically placed in the correct 
cells. 

Finally, we needed a mechanism to identify volunteers while ensuring 
their responses to the survey were anonymous. We accomplished this by 
creating a “confirmation” page in FrontPage. This page thanked the 
participants for participating in the survey and then asked them if they would 
be willing to volunteer for a follow-up phone interview. Those who 
volunteered clicked on a link that took them to a new FrontPage form that 
asked for their name and phone number. We did not gather date, time, and 
IP address because that information could have allowed us to match the 
date, submission time, and IP address to the same data in the original survey, 
violating their confidentiality. 

Thus, using a common software program, we were able to develop a 
complex survey that was tailored to the individual, maintained anonymity, 
allowed for quick and errorless entry of data, and allowed us to solicit 
volunteers for a follow-up telephone interview. Because the survey was 
anonymous, we could not control for multiple responses, although our 
experience is that people are much less likely to respond to a survey at all 
than to complete it more than once. 
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We also noted that persons needing assistance could contact the authors 
and the form would be filled out over the phone. Those requiring assistance 
did not need to give their names or identifying information to access this 
service. Because this was official agency business, they could use the agency 
phones for this call without cost to them. 

Implementation 

We “beta” tested the survey by completing the instrument ourselves 
from the perspective of each of the four employee groups and by answering 
every question, using all possible answers. This procedure checked for errors 
in coding and submission. We pilot tested the survey for errors in content, 
spelling, and grammatical syntax using a pre-selected group of agency 
employees. We also used this group to test the clarity of the instructions. 
These persons were instructed to take the survey “for real” and to note any 
errors or problems and report them to us. Although the agency used Windows 
98 and Internet Explorer 4.0 on almost all their computers, we tested the 
survey on the Macintosh and Windows 95 machines and with both Internet 
Explorer and Netscape Navigator 4.0. We also tested the form using JAWS 
test-to-speech synthesizer for persons with visual impairments. The survey 
did not render well in JAWS so we instituted the telephone alternative. 
Only one person used this service, however, as most staff with visual 
impairments completed the survey with the assistance of their colleagues. 
Based on the pilot study, small changes were made in the instructions and 
some syntax errors were corrected. No difficulty was observed with any 
browser. 

When the survey was finalized, notification was sent to the staff via 
the Internal e-mail system of the agency. Employees were instructed to go 
to the web site listed in the e-mail and complete the form. The initial 
announcement was sent at approximately 4:00 p.m. on a Thursday afternoon. 
After the initial announcement, we requested that additional reminders be 
sent periodically. Unfortunately, we could not access the agency e-mail 
system, so all our messages to the staff had to be relayed through two other 
persons. Moreover we were not part of the e-mail system so we could not 
verify if the e-mail reminders that we requested early in the process had 
been sent. We corrected the problem but not before over a week had passed. 

We did not have a central “help desk” but provided both our work and 
home phone numbers so that assistance was available during working hours 
every day the survey was open. We checked the data daily to ensure the 
data was being submitted and entered properly. 
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Analysis 

The total number of valid responses was 419 of the 1,136 persons to 
whom the survey announcement was sent, for an overall response rate of 
37%. Three of the four groups had response rates over 40% (46%, 45%, 
and 41%), but the fourth group had a response rate of only 16%. (We 
attributed this lower response rate to the diversity of job classifications 
within this category and the belief of persons within this category that the 
training needs survey was not relevant to them, although this is highly 
speculative.) The first group (with a 46% completion rate) had completed 
a paper needs assessment survey three years before and had a similar 
response rate (45%). The other groups had not been surveyed agency-wide 
before. 

As discussed above, in addition to the survey results, we gathered 
data on time and date of submission. Coupling this data with the reminder 
times and dates, we were able to graph response patterns for all our 
participants. These results are shown in Figures 1 through 3. Figure 1 
displays the time of day responding. Figure 2 gives the date (with day of 
the week) of response. Figure 3 shows the range of response times for each 
day and adds the median time of response. Reminders are indicated in bold. 
Double question marks (??) by a date designates when reminders were 
requested but their actual delivery could not be verified. Dates marked with 
an asterisk indicate the dates and times when reminders were sent and can 
be verified. 

Overall, it appears that response rates for three of the four groups 
were somewhat consistent with response rates for paper and pencil surveys, 
as determined by a meta-analysis of published research (Kerlinger, 1986) 
and consistent with meta-analytic results of web-based survey response 
rates (39.6%, according to Cook, Heath, and Thompson, 2000). For one 
employee group, response rates were consistent with an earlier paper and 
pencil survey for the same population. It also appears that reminders do 
influence response rates (see Figure 2). Moreover, Figure 1 suggests that 
staff were most likely to respond either when they first arrived at the office 
or just before they went home. Noting this trend, we reasoned that reminders 
should go out late in the day to catch people when they were most likely to 
respond immediately or so that the reminder would still be visible first 
thing in the morning, before the demands of the day began to impinge on 
the employee’s time. To test this assumption, we sent out a reminder on 
Wednesday, September 20, at 6:00 p.m., and another on Monday, September 
25, at 3:30 p.m. The response rate on September 21 was the highest of any 
give day (see Figure 2), and the median response time was 9:35 a.m. The 
response on September 25 was also encouraging with the median response 
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time of 4:15 p.m. In addition, the median response time for the following 
day was 9:23 a.m. The number of people completing the survey was lower 
on September 25 and 26, but this may have been caused because of saturation 
(i.e., those who were going to respond had done so by that time). 

Conclusion and Recommendations 

Form development 

We chose to create one form for all four employee groups. Given the 
large number of questions, this page would have loaded very slowly on a 
modem. In our situation all employees had access to high speed Internet 
connections, but researchers not having this luxury may consider breaking 
the form into its components and linking to the specific pages. By using 
one form, we also were able to link the general questions with the employee 
specific questions, giving us more information about each group. 

Depending on how the data is to be analyzed, saving the data in tab- 
delimited format allows easy importation into spreadsheet programs such 
as Excel. Moreover, if a statistical package such as SPSS is to be used, 
making all the field names conform to the SPSS naming convention will 
allow SPSS to retain the variable’s names. It should be noted that SPSS 
will import the data even if the variables are not named as SPSS mandates, 
but the variable names will be lost and replaced with generic names 
generated by SPSS, which will require re-entry by the researcher, a time 
consuming task that can easily be avoided. 

Reminders 

Although we had the severe limitation of not being able to control 
when reminders were sent at the beginning of the study, we were able to 
test our hypothesis about when was the best time to send reminders. From 
observation of the response patterns early in the study, we surmised that 
sending reminders late in the day or very early in the morning has the 
potential to be the most effective. We hasten to add that this hypothesis is 
as yet highly tentative for at least two reasons. First, the reminder of 
September 20 may have been the first reminder the staff received, 
notwithstanding the “bump” in responses on Tuesday, September 12, when 
we requested a reminder but did not receive confirmation that it was sent. 
If the September 20 reminder was the first, then perhaps those who had 
intended to complete the survey but had not done so now recognized that 
this was the time to carry out their intention. Second, we were not able to 
test sending out the reminder at other times of the day as we were near the 
end of the study. The response rates and median time of response do. 
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nonetheless, lend support to the hypothesis that reminders late in the day 
can lead to higher response rates. Our study may have some limited 
generalizability because the population was a “closed” group. That is, all 
participants worked for a single agency and could be contacted in a 
systematic and controlled manner, a luxury that many researchers will not 
have. 

Notwithstanding these limitations, this study provides researchers with 
suggestions for constructing instruments, tabulating the data, and increasing 
response rates. Hopefully, others can apply this information and web-based 
survey research can become more precise. 
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Figure 2: Response Rate by Day 
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Figure 3: Range and Median Response Times by Date 
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